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Abstract 

The problem of finding roots or solutions of a nonlinear partial differ- 
ential equation may be formulated as the problem of minimizing a sum 
of squared residuals. One then defines an evolution equation so that in 
the asymptotic limit a minimizer, and often a solution of the PDE, is 
obtained. The corresponding discretized nonlinear least squares prob- 
lem is an often met problem in the field of numerical optimization, and 
thus there exist a wide variety of methods for solving such problems. We 
review here Newton's method from nonlinear optimization both in a dis- 
crete and continuous setting and present results of a similar nature for the 
Levernberg-Marquardt method. We apply these results to the Ginzburg- 
Landau model of superconductivity. 

1 Introduction 

Consider the problem of finding a solution to the PDE 

F{Du) = (1) 

where for u in the Sobolev space H := H^'^{fl), Du — {DaU : \a\ < 1}. fl is 
assumed to be a bounded domain of dimension n with smooth boundary. F 
is a function from M."~^^ to K™ which is commonly referred to as a Nemistkii 
operator. Let L := L'^{fl). In order that F o D : H ^ [i]™ be Frechet 
difFerentiable it is sufficient that F be and satisfy the growth bound |F'(a;)| < 
c\x\ for X S K""*"^ and some c > (see [5]). One can propose to find a solution 
of ([!]) by solving the minimization problem 

Eiu) . 
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The proposed method would ideahy be an existence results, proving that a 
minimizer exists and is a solution of the PDE, and it should give a recipe for 
computing such a solution numerically as for most nonlinear problem closed 
form solutions are not possible. In order to be effective, the numerical method 
should emulate an iteration in the infinite-dimensional Sobolev space in which 
the PDE is formulated. This formulation follows naturally from Neuberger's 
theory of Sobolev gradients. The Frechet derivative E'{u), which is a bounded 
linear functional on H, is represented by an element of H. This element is the 
Sobolev gradient of at u and is denoted by VhE{u): 

E'{u)h:=:: {h,VHE{u))H, h^H. 

Note that the gradient depends on the inner product attached to H. One 
considers the evolution equation 

z{0) = zoeH and z'{t) = HE{z{t)), t > 0. (3) 

The energy E is non-increasing on the trajectory z. Existence, uniqueness, and 
asymptotic convergence to a critical point are established by the following two 
theorems taken from [51 Chapter 4]. 

Theorem 1. Suppose that E is a non-negative real-valued function on a 
Hubert space H with a locally Lipschitz continuous Sobolev gradient. Then for 
each zq £ H there is a unique global solution of ([s]). 

Definition 1. The energy functional E satisfies a gradient inequality on K C 
H if there exists 9 € (0, 1) and m > so that for all x G K 

\\VhE{x)\\h > mE{xf. 

Theorem 2. Suppose that E is a non-negative functional on H with a 
locally Lipschitz continuous gradient, z is the unique global solution of and 
E satisfies a gradient inequality on the range of z. Then limj_j.oo z{t) exists 
and is a zero of the gradient, where the limit is defined by the H-norm. By the 
gradient inequality, the limit is also a zero of E. 

The above theorems provide a firm theoretical basis for the numerical treat- 
ment of a system of nonlinear PDE's by a gradient descent method that emu- 
lates ([3]); i.e., discretization in time and space results in the method of steepest 
descent with a discretized Sobolev gradient. Note that the Sobolev gradient 
method differs from methods based on calculus of variations in which the Euler- 
Lagrange equation is solved. Forming the Euler-Lagrange equation requires 
integration by parts to obtain the element that represents E'{u) in the inner 
product. This gradient is usually only defined on a Sobolev space of higher 
order than that of H. Hence, unlike the Sobolev gradient, the gradient is 
only densely defined on the domain of E. For gradient flows involving the 
gradient, existence and uniqueness results similar to those of Theorems [T] and 
(|2| may be proved, but only under stricter assumptions. 
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2 Newton and Variable Newton methods 



In order to solve the minimization problem ([2]), we take the first variation to 
obtain 

E'(u)h^ {F'{Du)Dh,F[Du))L. (4) 

We seek an evolution equation so that in the limit as time goes to infinity, we 
can find a zero. A natural setting would be if such an evolution came from a 
gradient system as defined in |2l. In particular, first assume that G{u) := F{Du) 
and G' {u) is invertible for each u. Then Newton's method 

m(0) = Mo and u'{t) = ^{G' {u))-^G{u) 

is the gradient system associated with the inner product 

{'",w)g^(^u) ^ {G'{u)v,G'{u)w)l- (5) 

This is achieved by noting that 

E'{u)h^{G'{u)h,G\u){G'{u))-^G{u)) = {h,{G' {u))-^G{u)) g.^^y 

Continuous Newton's method gives an infinite dimensional method for find- 
ing solutions of ([T]) ; see, e.g., [1], [I], and [10] for zero-finding results of Nash- 
Moser type ([8]). In [5] Newton's method is discussed in relation to gradient 
descent methods. It is shown that, while the method of steepest descent is 
locally optimal in terms of the descent direction for a fixed metric, Newton's 
method is optimal (in a sense which is made precise) in terms of both the di- 
rection and the inner product in a variable metric method. When Newton's 
method is available the quadratic rate of convergence to a solution makes this 
method ideal in a numerical setting. 

For many partial differential equations G"(u) may not be invertible for all u 
thus Newton's method cannot be applied in the infinite dimensional setting. In 
the finite dimensional setting, one needs that the initial condition be close to 
the solution to obtain convergence. Another option is to minimize ([2| using a 
variable metric method. We give here the description of one such method which 
when discretized gives a variation of the Levenberg-Marquardt method. The 
results are taken from 7J. 

For u E H = H^{n) consider the bilinear form on H defined by 

{v, w)u = {v, w)h + {l/\{u)){G'{u)v, G'{u)w)l (6) 

where A(m) is a positive damping parameter. By our assumption that G'{u) G 
L{H,L), there exists a constant c = c{u) so that for all v E H, \\G'{u)v\\l < 
c\\v\\h. Hence \\v\\l = Ml + (l/A(«))|lG'H«||i, and 

\\v\\h < \\v\\u < v/1 + c(u) \\v\\h 

so that, for each u € i/, (|6| defines a norm that is equivalent to the standard 
Sobolev norm on H. The gradient of E with respect to (•, •)„ is defined to be 
the unique element VuE{u) so that 

E'{u)h = {h, VuE{u))u y h£H. (7) 
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Consider the gradient flow 



z{0) ^ uo e H and z'{t) ^ -V^^t)E{z{t)), t>0. (8) 

We seek a solution z of this flow so that Uf = \imt~yoo z{t) exists and 
E'{uf) = 0. In the case that a gradient inequality is satisfied, a zero of the 
derivative of E is also a zero of E and hence a solution of ([l]). The key idea 
in obtaining global existence and asymptotic convergence of the flow ([8| is to 
obtain an expression for the abstract gradient. We obtain this expression by 
considering a family of orthogonal projection onto the graph of a closed densely 
defined operator. In particular, let S'„ : — > be given by Su = {^) 

where Th = {Dah : \a\ — 1} and T^h = {1 / X{u))G' {u)h. Since the domain 
of Su is all of H, Su can be viewed as a densely defined operator on L. It is 
also the case that Su is a bounded linear operator from H to [L]""*"^. Note that 
Su need not be bounded when viewed as an operator from L to [L]"+^. It also 
follows that the graph of Su is a closed subspace of [L]"+^. By a theorem of von 
Neumann ([13]) there exists a unique orthogonal projection from onto 
the graph of Su, and the projection is given by 



Pu 



{I + S*Su) ^ Su{I + SuSu) ^ 

Su{I + ^u^u) I ^ (I ^ SuSu) 



This result can also be found in W,, Theorem 5.2]. Here S** is the adjoint of Su 

with Su treated as a closed and densely defined operator on L, and hence S* is 

also closed and densely defined on its domain . Note also that 

and + S'^S'*)"^ are everywhere defined on L and [L]""*"^, respectively, and 

are bounded as operators from L to L and [L]"+^ to L, respectively (QTl Sec 

118]). 

2.1 An expression for the gradient 

We will obtain an expression for the gradient given in ([7|. The graph of Su is 

{(t^) ■ ^ ^ -^j- Since P„ is the unique orthogonal projection of [i]"+^ onto 

the graph of Su, Pu is the identity on the graph of Su, and thus Pu{x\) — (t 
for all h (z H . Using symmetry of P„, we have 

E'{u)h= {G'{u)h,G{u))L = 
Dh \ / 



^"^\V(l/yAM)G'H/.;'VGH 
^(''"((l/yAM)G'H/^)'(G(.) 
^ ( {{l/^)G'{u)h) ' ^" {gIu)J I 





G{u\ 
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where 11 is the operator that extracts the first element of a pair: n(p = x. For 
each M G iJ we have a gradient 



(9) 



Theorem 3. 

VuE{u) = v/aRmT„*(/ + TuMT*)-^F{Du), (10) 

where T* is the adjoint of r„ when viewed as a closed and densely defined 
operator on , and M = (/ + T*T)^^ = {D* D)^^ is a smoothing operator. 

A steepest descent iteration with a discretization of this gradient is a gener- 
ahzed Levenberg-Marquardt iteration in which the identity or a diagonal matrix 
is replaced by the positive definite operator M^^ ^ D*D. 

The generalized Levenberg-Marquardt method is given by 

u„+i = u„ - X{u){X{u)D*D + {G\u)yG'{u))-\G'{u))*F{Du) 

which is a forward Euler discretization of Q with time step 1. The expression 
of the gradient was used to obtain the following results. 

Theorem 4. Suppose E is as defined in ^ with FoD a G^ function defined on 
H with range in , and suppose that A : _ff — >■ M is locally Lipschitz continuous 
and bounded below by a positive constant. Then the gradient system ([8| has a 
unique global solution z G G^{[0,oo), H). 



Theorem 5. Suppose that there exists ^ e (0, 2) so that if u G H . there is 
li 



7(u) > such that for each g in the domain ofG'{u)* with \\g\\L = 1 



the linear PDF 

G'{u)x = g 

has a solution x G H with < z^jyj- Then a gradient inequality is satis- 

fied. Here G'{u)* denotes the adjoint of G'{u) when viewed as a densely define 
operator on L. 

Theorem 6. Suppose that the hypotheses of Theorem^are satisfied so that (|8| 
has a unique global solution z, and suppose that the hypotheses of TheorernJ5\ 
are satisfied on an open region containing the range of z. Then u — limt^aoz(t) 
exists and F{Du) — 0. 



3 Applications to superconductivity 

In [S], we applied a variation of the above formulation to study the Ginzburg- 
Landau energy. The Ginzburg-Landau model postulates that the behavior of 
the superconducting electrons in materials can be described by a complex valued 
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wave function in which case the Ginzburg-Landau (Gibbs free) energy is given 

by 



E{u,A) 



\Vu~iAu\^ iVxA- 



+ 



1) 



(11) 



in nondimensionaUzed form. Here u is the complex valued wave function which 
gives the probability density of the superconducting electrons, A is the in- 
duced magnetic vector potential, Hq is the external magnetic field, and k is 
the Ginzburg-Landau coefficient which characterizes the type of superconduct- 
ing sample (type I or type II) . The central Ginzburg-Landau problem is to find a 
minimizer of the Ginzburg-Landau energy. Note that this energy can be written 
in the form ([2| with 



F{D{u,A))^ 



ri 
si 



as 
ar 



\ 



(12) 



r2 + bs 
S2 - br 
6i - a2 - Hq 

for u = Q and A^ One can check that FoDisC^ from H = [H^{fl)]^ to 
L = [L^(r2)]^. Thus the gradient system ([s]) has a unique global solution when 
A satisfies the properties of Theorem |4] We studied the rate of convergence of 
this fiow to a minimizer using a trust-region method in [6]. In future work, it 
would be a very nice result to obtain a gradient inequality to prove convergence 
by verifying the condition of Theorem |5] Since for the Ginzburg-Landau energy, 
a minimizer does not correspond to a zero of the energy, the definition of the 
gradient inequality is altered to the following definition taken from [2]. 

Definition 2. Suppose E is as in ([2| and that E achieves a local minimum at 
Um ■ Then E is said to satisfy a gradient inequality in a neighborhood of Um if 
there exists a ball B containing it,„ and ^ £ (0, 1), c > such that all v £ B 

\E{v)-E{u„,)\^ <c\\V.,E{v)\l. 

This formulation was used to obtain a stabilization result for the Ginzburg- 
Landau equations in [3] and [T^]. In figure [Tj we give contour plots of minimizers 
for various parameters. 



4 Conclusion 

We extended the theory of Sobolev gradients to include gradients associated with 
a variable inner product, and we described a generalized Levenberg-Marquardt 
method as a gradient flow in an infinite-dimensional Sobolev space. We pre- 
sented conditions under which the flow is guaranteed to converge to a zero of 
a residual representing a solution of a nonlinear partial differential equation. 
The conditions include smoothness of the residual and satisfaction of a gradient 
inequality. 
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Figure 1: Vortex configurations corresponding to a density plot of the minimizer 
for K = 4 and Hq = 4, 4, 6, 8. 

Our results provide a theoretical basis for a practical and effective method 
of solving nonlinear partial differential equations. As a further development we 
will seek to apply our results to particular problems. A proof that a numerical 
iteration has a convergent counterpart in the infinite-dimensional setting rep- 
resents an important contribution to the goal of a unified approach to treating 
partial differential equations by numerical and analytical methods. 
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